Repository navigation
Deploy the testing property of pipelines and show remote changes in bundle plan - #7438
tharunbhuk-db wants to merge 10 commits into
Conversation
|
Commit: Regression test reportSelected 4 changed tests (limit=20, 0 not selected) 8 tests: regression=8 unreleased=0 coverage=0 failing=0
TestAccept/bundle/environments/dependencies/DMS=: branch ✅ | main (0f6d067) ❌ | latest (v1.26.0) ❌main (0f6d067): latest (v1.26.0): same output as main (0f6d067) TestAccept/bundle/environments/dependencies/DMS=true: branch ✅ | main (0f6d067) ❌ | latest (v1.26.0) ❌main (0f6d067): latest (v1.26.0): same output as main (0f6d067) TestAccept/bundle/resources/pipelines/testing-dynamic-version/DMS=: branch ✅ | main (0f6d067) ❌ | latest (v1.26.0) ❌main (0f6d067): latest (v1.26.0): TestAccept/bundle/resources/pipelines/testing-dynamic-version/DMS=true: branch ✅ | main (0f6d067) ❌ | latest (v1.26.0) ❌main (0f6d067): latest (v1.26.0): TestAccept/bundle/resources/pipelines/testing-same-name-libraries/DMS=: branch ✅ | main (0f6d067) ❌ | latest (v1.26.0) ❌main (0f6d067): latest (v1.26.0): TestAccept/bundle/resources/pipelines/testing-same-name-libraries/DMS=true: branch ✅ | main (0f6d067) ❌ | latest (v1.26.0) ❌main (0f6d067): latest (v1.26.0): TestAccept/bundle/resources/pipelines/testing/DMS=: branch ✅ | main (0f6d067) ❌ | latest (v1.26.0) ❌main (0f6d067): latest (v1.26.0): TestAccept/bundle/resources/pipelines/testing/DMS=true: branch ✅ | main (0f6d067) ❌ | latest (v1.26.0) ❌main (0f6d067): latest (v1.26.0): |
❌ Integration test reportCommit: 9652944
18 interesting tests: 17 FAIL, 1 RECOVERED
Top 3 slowest tests (at least 2 minutes):
|
48e4a95 to
fd9fedb
Compare
1fd5e11 to
3af4a58
Compare
fd9fedb to
b10a21e
Compare
3af4a58 to
7498e1b
Compare
b10a21e to
e5ff44c
Compare
7498e1b to
e79333b
Compare
| trace print_requests.py //api/2.0/pipelines --nostamp | jq '{method, path, testing: .body.testing}' | ||
|
|
||
| title "No changes after the deploy\n" | ||
| trace $CLI bundle plan |
There was a problem hiding this comment.
Just to double check: on dogfood, did bundle plan show 0 changes right after deploying a pipeline with testing? The description covers existing pipelines and removing it, but the fake server echoes the body back, so it can't tell us whether the backend normalizes glob.include or the -e /Workspace/...[test] dependency on read.
There was a problem hiding this comment.
Yes. Pranshu checked it on a real workspace (see his review on #7300): after a deploy with testing, the next plan showed 0 changes. The fake server cannot show this, as you say.
| } | ||
|
|
||
| func (r *ResourcePipeline) DoRead(ctx context.Context, id string) (*PipelineRemote, error) { | ||
| if env.ExperimentalPipelineTesting(ctx) { |
There was a problem hiding this comment.
Why gate the read on the env var when create and update are gated on config (config.Testing != nil)? The raw GET is the same request, so always reading spec.testing would be free, and as is two people running bundle plan on a pipeline that got testing added in the UI see different plans depending on their env.
There was a problem hiding this comment.
This is out of date. This PR has no raw read any more: the Go SDK reads spec.testing in DoRead for everyone, with or without the variable, as main already does.
|
|
||
| // doRaw sends a request that the Go SDK cannot express yet because it lacks the testing field. | ||
| func (r *ResourcePipeline) doRaw(ctx context.Context, method, path string, request, response any) error { | ||
| apiClient, err := client.New(r.client.Config) |
There was a problem hiding this comment.
nit: this builds a new API client on every call, and DoRead runs once per pipeline per plan. Can we create it once in New like the cmd/air callers do?
There was a problem hiding this comment.
This is out of date. The raw client is gone; Create, Edit and Get use the SDK client.
| update_file.py databricks.yml "$(sed -n '/^ testing:/,$p' databricks.yml)" "" | ||
| trace $CLI bundle plan | ||
| trace $CLI bundle deploy | ||
| trace print_requests.py //api/2.0/pipelines --nostamp | jq '{method, path, testing: .body.testing}' |
There was a problem hiding this comment.
nit: null here can mean either an absent key or "testing": null in the body. .body | has("testing") would make the assertion explicit.
There was a problem hiding this comment.
Done in 7401411. The step now prints has_testing: (.body | has("testing")), which is false.
e79333b to
35031d3
Compare
858d4df to
52dc912
Compare
35031d3 to
2126212
Compare
52dc912 to
650fc96
Compare
2126212 to
d98a141
Compare
650fc96 to
2189384
Compare
d98a141 to
7401411
Compare
2189384 to
1c74f67
Compare
7401411 to
36406cf
Compare
36406cf to
4a61012
Compare
Add resources.pipelines.<key>.testing, with the test sources and test-only dependencies that databricks pipelines test runs with. The field comes from the Go SDK's CreatePipeline. The CLI adds its own descriptions for it. The property is experimental. It needs DATABRICKS_BUNDLE_ENABLE_EXPERIMENTAL_PIPELINE_TESTING=true. While the variable is off, bundle validate rejects testing and names the variable. Bundles without testing are not changed. Later PRs translate its paths. Co-authored-by: Isaac <no-reply@databricks.com>
The check walks resources.pipelines.*.testing with dyn.MapByPattern, like the check of the development property, which gives the path and the locations directly. The acceptance test covers the error and the enabled case, so the unit test is removed. Co-authored-by: Isaac <no-reply@databricks.com>
The Go SDK now has the testing field, so the property needs no separate opt-in. Co-authored-by: Isaac <no-reply@databricks.com>
Relative paths in testing.sources and testing.environment.dependencies resolve to their deployed workspace paths, like libraries and environment dependencies. Local wheels in test dependencies are uploaded like other libraries, and the same-name check covers them. The testing property is only accepted with DATABRICKS_BUNDLE_ENABLE_EXPERIMENTAL_PIPELINE_TESTING=true, so bundles without it are not changed. Co-authored-by: Isaac <no-reply@databricks.com>
environment.dependencies and testing.environment.dependencies share the translate modes and skip functions, so the patterns are built in one loop over the two locations. The order of the patterns does not change. Co-authored-by: Isaac <no-reply@databricks.com>
With dynamic_version, the deploy uploaded the patched wheel, but testing.environment.dependencies kept the original file name, so tests could run cached code. Co-authored-by: Isaac <no-reply@databricks.com>
Co-authored-by: Isaac <no-reply@databricks.com>
dist/ is ignored, so the test now copies the wheel from the artifacts tests. Co-authored-by: Isaac <no-reply@databricks.com>
…emote changes in bundle plan bundle deploy sends a pipeline's testing on Create and Edit, and bundle plan reads it back, so a change made outside the bundle shows in the plan and the next deploy undoes it. The Go SDK has the field, so the direct engine and the test server already handle it. This change adds the acceptance test for it. Co-authored-by: Isaac <no-reply@databricks.com>
The last step of the acceptance test now asserts that the request body has no testing key, instead of printing testing as null, which can also mean an explicit null. Co-authored-by: Isaac <no-reply@databricks.com>
1706c59 to
669ee9b
Compare
4a61012 to
9652944
Compare
pietern
left a comment
There was a problem hiding this comment.
This PR only appends to an existing acc test.
If coverage was already there, the PR can be dropped.
669ee9b to
e036701
Compare
|
Agreed that remote-edit drift, updates and removal behave like any other SDK field now, so I dropped those sections. I kept only the deploy-request check, folded into #7437: @andrewnester asked for it there (#7437 (comment)), and it covers code that is new in that PR (the test wheel upload and the rewrite of |
🥞 Stacked PR
bundle deploysendstesting, andbundle planshows a change made to it outside the bundle; the next deploy undoes it.Tests: acceptance
bundle/resources/pipelines/testing(deploy, plan after a remote edit, update, removal).This pull request and its description were written by Isaac.